The data set has information about features of silhouette extracted from the images of different cars¶
Four "Corgie" model vehicles were used for the experiment: a double decker bus, Cheverolet van, Saab 9000 and an Opel Manta 400 cars. This particular combination of vehicles was chosen with the expectation that the bus, van and either one of the cars would be readily distinguishable, but it would be more difficult to distinguish between the cars.
1. Read the dataset using function .dropna() - to avoid dealing with NAs as of now¶
In [194]:
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy.stats import zscore
from sklearn.preprocessing import LabelEncoder
from sklearn.cluster import KMeans
from scipy.spatial import distance
from scipy.cluster.hierarchy import cophenet, dendrogram, linkage
from scipy.spatial.distance import pdist #Pairwise distribution between data points
from scipy.cluster.hierarchy import fcluster
In [13]:
silhouette_df = pd.read_csv("vehicle.csv")
silhouette_df.sample(10)
Out[13]:
In [11]:
silhouette_df.shape
Out[11]:
In [4]:
silhouette_df.info()
In [18]:
silhouette_df.dropna(inplace=True)
In [19]:
silhouette_df.info()
2. Print/ Plot the dependent (categorical variable) - Class column¶
Since the variable is categorical, you can use value_counts function
In [20]:
silhouette_df['class'].value_counts()
Out[20]:
In [21]:
sns.countplot(x='class', data=silhouette_df)
Out[21]:
In [37]:
plt.figure()
pd.Series(silhouette_df['class']).value_counts().sort_index().plot(kind = 'bar')
plt.ylabel("Count")
plt.xlabel("class")
plt.title('Number of class');
Check for any missing values in the data¶
In [22]:
silhouette_df.isnull().any()
Out[22]:
In [26]:
silhouette_df.isnull().sum()
Out[26]:
In [36]:
sns.pairplot(silhouette_df, diag_kind='kde' , hue = 'class')
Out[36]:
3. Standardize the data¶
In [31]:
silhouette_df.dtypes
Out[31]:
In [71]:
# Class feature is an object column , so it should not be standardized
silhouette_df_X = silhouette_df.drop('class',axis=1)
Since the dimensions of the data are not really known to us, it would be wise to standardize the data using z scores before we go for any clustering methods. You can use zscore function to do this
In [72]:
silhouette_df_z = silhouette_df_X.apply(zscore)
In [73]:
labelencoder=LabelEncoder()
#silhouette_df_z['class'] = labelencoder.fit_transform(silhouette_df['class'])
In [89]:
silhouette_df_z.sample(2)
Out[89]:
In [90]:
sns.pairplot(silhouette_df_z, diag_kind='kde')
Out[90]:
K - Means Clustering¶
4. Plotting Elbow/ Scree Plot¶
In [76]:
cluster_range = range(2,10)
cluster_errors = []
distortion = []
for cluster_n in cluster_range :
clusters = KMeans(cluster_n, n_init = 5)
clusters.fit(silhouette_df_z)
labels = clusters.labels_
cluster_errors.append(clusters.inertia_)
distortion.append(sum(np.min(distance.cdist(silhouette_df_z, clusters.cluster_centers_, 'euclidean'), axis=1))/ silhouette_df_z.shape[0])
clusters_df = pd.DataFrame( { "num_clusters":cluster_range, "cluster_errors": cluster_errors } )
clusters_df[0:15]
Out[76]:
In [77]:
# Elbow plot
plt.figure(figsize=(12,6))
plt.plot( clusters_df.num_clusters, clusters_df.cluster_errors, marker = "o" )
Out[77]:
Use Matplotlib to plot the scree plot - Note: Scree plot plots distortion vs the no of clusters
In [78]:
plt.plot(range(2,10), distortion, 'go-')
Out[78]:
Find out the optimal value of K¶
Optimal value of k is 3 As per analysing pair plot and elbow plot (bend is maximum at 3)¶
In [ ]:
Using optimal value of K - Cluster the data.¶
Note: Since the data has more than 2 dimension we cannot visualize the data. As an alternative, we can observe the centroids and note how they are distributed across different dimensions
In [84]:
cluster = KMeans( n_clusters = 3, random_state = 100 )
cluster.fit(silhouette_df_z)
silhouette_df_z_copy = silhouette_df_z.copy(deep = True) # Creating a mirror copy for later re-use instead of building repeatedly
You can use kmeans.clustercenters function to pull the centroid information from the instance
In [92]:
centroids = cluster.cluster_centers_
centroids
Out[92]:
5. Store the centroids in a dataframe with column names from the original dataset given¶
Hint: Use pd.Dataframe function
In [93]:
centroid_df = pd.DataFrame(centroids, columns = list(silhouette_df_z) )
centroid_df
Out[93]:
In [95]:
prediction=cluster.predict(silhouette_df_z)
silhouette_df_z["PREDICTED_CLASS"] = prediction
In [96]:
silhouette_df_z.sample(5)
Out[96]:
Use kmeans.labels_ function to print out the labels of the classes¶
In [98]:
cluster.labels_
# can also use silhouette_df_z['PREDICTED_CLASS']
Out[98]:
In [ ]:
Hierarchical Clustering¶
6. Variable creation¶
For Hierarchical clustering, we will create datasets using multivariate normal distribution to visually observe how the clusters are formed at the end
In [139]:
np.random.seed(101)
a = np.random.multivariate_normal([10, 0], [[3, 1], [1, 4]], size=[100,])
b = np.random.multivariate_normal([0, 20], [[3, 1], [1, 4]], size=[50,])
c = np.random.multivariate_normal([10, 20], [[3, 1], [1, 4]], size=[100,])
In [140]:
print(a[:5])
print(b[:5])
print(c[:5])
In [142]:
print(a.shape)
print(b.shape)
print(c.shape)
7. Combine all three arrays a,b,c into a dataframe¶
In [141]:
hier_clustering_data = np.concatenate((a,b,c),axis=0)
hier_clustering_data[:5]
Out[141]:
In [143]:
hier_clustering_df = pd.DataFrame( hier_clustering_data,columns=["A","B"] )
hier_clustering_df.head()
Out[143]:
In [144]:
hier_clustering_df.shape
Out[144]:
In [ ]:
8. Use scatter matrix to print all the 3 distributions¶
In [137]:
sns.pairplot(hier_clustering_df, diag_kind='kde')
Out[137]:
In [148]:
plt.scatter(x=hier_clustering_df['A'],y=hier_clustering_df['B'])
Out[148]:
9. Find out the linkage matrix¶
Use ward as linkage metric and distance as Eucledian
In [162]:
# cophenet index is a measure of the correlation between the distance of points in feature space and distance on dendrogram
# closer it is to 1, the better is the clustering
link_z = linkage(hier_clustering_df,'ward')
c, coph_dists = cophenet(link_z , pdist(hier_clustering_df))
c
Out[162]:
In [ ]:
In [ ]:
10. Plot the dendrogram for the consolidated dataframe¶
In [167]:
plt.figure(figsize=(8, 8))
plt.title('Agglomerative Hierarchical Clustering Dendogram')
plt.xlabel('sample index')
plt.ylabel('Distance')
dendrogram(link_z, leaf_rotation=90.,color_threshold = 50, leaf_font_size=8. )
plt.tight_layout()
In [ ]:
In [ ]:
11. Recreate the dendrogram for last 3 merged clusters¶
In [ ]:
Hint: Use truncate_mode='lastp' attribute in dendrogram function to arrive at dendrogram
In [230]:
dendrogram(
link_z,
color_threshold=81,
truncate_mode='lastp', # show only the last p merged clusters
p=3, # show only the last p merged clusters
)
plt.show()
In [ ]:
12. From the truncated dendrogram, find out the optimal distance between clusters which u want to use an input for clustering data¶
In [206]:
# From the truncated dendrogram the optimal distance between clusters is 80
In [ ]:
13. Using this distance measure and fcluster function to cluster the data into 3 different groups¶
In [210]:
labels = fcluster(link_z,t=80,criterion='distance')
labels
Out[210]:
In [ ]:
Use matplotlib to visually observe the clusters in 2D space¶
In [211]:
plt.scatter(x=hier_clustering_df['A'],y=hier_clustering_df['B'],c=labels)
Out[211]:
In [ ]: